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Abstract 

The incompressibility method is an elementary yet powerful proof technique. It 
has been used successfully in many areas To further demonstrate its power and 

elegance we exhibit new simple proofs using the incompressibility method. 

1 Introduction 

The incompressibility of individual random objects yields a simple but powerful proof tech- 
nique: the incompressibility method. This method is a general purpose tool that can be 
used to prove lower bounds on computational problems, to obtain combinatorial properties 
of concrete objects, and to analyze the average complexity of an algorithm. Since the early 
1980's, the incompressibility method has been successfully used to solve many well-known 
questions that had been open for a long time and to supply new simplified proofs for known 
results. A survey is [pd]| . 

The purpose of this paper is pragmatic, in the same style as |12| , and a companion 
paper Q. We want to further demonstrate how easy the incompressibility method can be 
used, via a new collection of simple examples. The proofs we have chosen to be included 
here are not difficult ones. They are from diverse topics. Most of these are well-known 
topics such as sorting. Some results are new (but this is not important) such as curve 
fitting lower bound, and some results were known before. In all cases, the new proofs are 
much simpler than the old ones (if they exist). 
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2 Kolmogorov Complexity and the Incompressibility Method 



We use the following notation. Let x be a finite binary string. Then l(x) denotes the length 
(number of bits) of x. In particular, 1(e) = where e denotes the empty word. 

We can map {0, 1}* one-to-one onto the natural numbers by associating each string with 
its index in the length-increasing lexicographical ordering 

(e, 0), (0, 1), (1, 2), (00, 3), (01, 4), (10, 5), (11, 6), . . . . (1) 

This way we have a binary representation for the set of all natural numbers that is different 
from the standard binary representation. It is convenient not to distinguish between the 
first and second element of the same pair, and call them "string" or "number" arbitrarily. 
As an example, we have 1(7) = 00. Let x, y, £ M, where M denotes the natural numbers. Let 
To, Ti, ... be a standard enumeration of all Turing machines. Let (•, •) be a standard one-one 
mapping from J\f x J\f to M, for technical reasons chosen such that l({x, y)) = l(y) + 0(l(x)). 



Informally, the Kolmogorov complexity, [ 14 ] , of x is the length of the shortest effective 



description of x. That is, the Kolmogorov complexity C(x) of a finite string x is simply the 
length of the shortest program, say in FORTRAN (or in Turing machine codes) encoded in 
binary, which prints x without any input. A similar definition holds conditionally, in the 
sense that C(x\y) is the length of the shortest binary program which computes x on input 
y. Kolmogorov complexity is absolute in the sense of being independent of the program- 
ming language, up to a fixed additional constant term which depends on the programming 
language but not on x. We now fix one canonical programming language once and for all 



as reference and thereby C(). For the theory and applications, as well as history, see 11] 
A formal definition is as follows: 



Definition 1 Let U be an appropriate universal Turing machine such that 

U(((i,p),y))=T i ((p,y)) 

for all i and (p, y) . The conditional Kolmogorov complexity of x given y is 

C{x\y) = min {l(p) : U((p,y}) = x}. 
pe{o,i|* 

The unconditional Kolmogorov complexity of x is defined as C(x) := C(x\e). 

It is easy to see that there are strings that can be described by programs much shorter 
than themselves. For instance, the function defined by f(l) = 2 and f(i) = 2^ l ~ 1 ) for 
i > 1 grows very fast, f(k) is a "stack" of k twos. Yet for each k it is clear that f(k) has 
complexity at most C(k) + 0(1). 

By a simple counting argument one can show that whereas some strings can be enor- 
mously compressed, the majority of strings can hardly be compressed at all. For each n 
there are 2 n binary strings of length n, but only J27=o ^ = 2 n — 1 possible shorter descrip- 
tions. Therefore, there is at least one binary string x of length n such that C(x) > n. We 
call such strings incompressible. It also follows that for any length n and any binary string 
y, there is a binary string x of length n such that C(x\y) > n. 
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Definition 2 For each constant c we say a string x is c-incompressible if C(x) > l(x) — c. 

Strings that are incompressible (say, c-incompressible with small c) are patternless, 
since a pattern could be used to reduce the description length. Intuitively, we think of such 
patternless sequences as being random, and we use "random sequence" synonymously with 
"incompressible sequence." It is possible to give a rigorous formalization of the intuitive 
notion of a random sequence as a sequence that passes all effective tests for randomness, 
see for example |Tl|| . 

How many strings of length n are c-incompressible? By the same counting argument we 
find that the number of strings of length n that are c-incompressible is at least 2 n — 2 n ~ c + 1 . 
Hence there is at least one O-incompressible string of length n, at least one-half of all 
strings of length n are 1-incompressible, at least three-fourths of all strings of length n are 
2-incompressible, . . . , and at least the (1 — l/2 c )th part of all 2" strings of length n are 
c-incompressible. This means that for each constant c > 1 the majority of all strings of 
length n (with n > c) is c-incompressible. We generalize this to the following simple but 
extremely useful Incompressibility Lemma. 

Lemma 1 Let c be a positive integer. For each fixed y, every set A of cardinality m has at 
least m{\ — 2~ c ) + 1 elements x with C{x\y) > [k>g?nj — c. 

Proof. By simple counting. □ 
Note that obviously, for some c, C(x\y, A) < [logmj + c. 

As an example for the lemma, set A = {x : l(x) = n}. Then the cardinality of A is 
m = 2 n . Since it is easy to assert that C(x) < n + c for some fixed c and all x in A, Lemma |l] 
demonstrates that this trivial estimate is quite sharp. The deeper reason is that since there 
are few short programs, there can be only few objects of low complexity. 

Definition 3 A prefix set, or prefix-free code, or prefix code, is a set of strings such that 
no member is a prefix of any other member. A prefix set which is the domain of a partial 
recursive function (set of halting programs for a Turing machine) is a special type of prefix 
code called a self- delimiting code because there is an effective procedure which reading 
left-to-right determines where a code word ends without reading past the last symbol. A 
one-to-one function with a range that is a self-delimiting code will also be called a self- 
delimiting code. 

A simple self-delimiting code we use throughout is obtained by reserving one symbol, 
say 0, as a stop sign and encoding a natural number x as 1^0. We can prefix an object with 
its length and iterate this idea to obtain ever shorter codes: 



Thus, Ei(x) = l l ^0x and has length l{Ex{x)) = 2l(x) + 1; E 2 (x) = Ei(l(x))x and has 
length l(E2(x)) = l(x) + 2l(l(x)) + 1. We have for example 




for i = 0, 
for i > 0. 



(2) 



l(E 3 (x)) < l{x) +logZ(x) + 21oglogZ(x) + 1. 
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Define the pairing function 

(x,y) = E 2 {x)y 



(3) 



with inverses (-)i, (')%. This can be iterated to ((•, •), •). 

In a typical proof using the incompressibility method, one first chooses an individually 
random object from the class under discussion. This object is effectively incompressible. 
The argument invariably says that if a desired property does not hold, then the object 
can be compressed. This yields the required contradiction. Then, since most objects are 
random, the desired property usually holds on average. 



3 Lower Bound for Sorting 

We begin this paper with a very simple incompressibility proof for a well-known lower bound 
on comparison based sorting. 

Theorem 1 Any comparison based sorting algorithm requires O(nlogn) comparisons to 
sort an array of n elements. 

Proof. Let A be any comparison based sorting algorithm. Consider permutation I of 
{1, . . . , n} such that 

C(I\A,P) > logn! 

where P is a fixed program to be defined. Suppose A sorts I in m comparisons. We can 
describe / by recording the binary outcomes of the m comparisons, which requires a total 
of m bits. Let P be such a program converting m to /. Thus, 

m > C(I\A,P) > logn! 

Hence, m > logn! = nlogn — 0(n). 

□ 

The above proof also easily implies a lower bound of nlogn — 0(n) on the average 
number of comparisons required for sorting. 



4 Space Filling Curves 

In Niedermeier, Reinhardt, and Sanders studied the following problem: In an n x n 
mesh, consider a computable curve fitting scheme that maps the numbers from {1, . . . , n 2 } 
into the mesh, each number occupying one spot in the mesh. Many algorithms in parallel 
computing, computational geometry, and image processing depend on "locality-preserving" 



indexing scheme for meshes. [16] has shown that for any indexing scheme, there exist a pair 
i and j such that 



d(i,j) > sJS.5\i -j\-l 

where d is Euclidean distance. (When d is other distances, like Manhattan or Z^, Kol- 
mogorov complexity argument works similarly.) However, it is much more interesting to 
obtain an "average-case" bound, both theoretically and practically. The question for the 
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average-case is open. In fact, many experiments have been performed by researchers in or- 
der to determine the average distance [16]. We prove such a bound here with much simpler 
argument using the incompressibility method. 



Theorem 2 d(i,j) > y/2.5\i — j\ for many pairs of i,j's. 

Proof. Let N = n 2 . Consider a computable curve-fitting scheme F. Let's assume that 
F puts i in a corner. Consider j's such that 

C(j\i)> log N. (4) 

We know that there is a constant c > such that for every N there are N/c such j's 
(if there exists one such j, then there exist 1/c portion of them by the argument used in 
Exercise 2.2.6, p. 117 in Q). 

Also, we can argue that \i — j\ < N/2 for at least half of the j's. For if this is not the 
case, we can change the universal TM in the definition of Kolmogorov complexity by just 
making the new universal TM printing (1) whenever the old universal TM prints 1 (0). 
Then for each j, let j' be the l's complement of j, we have either 

\i-j\ <N/2, 



or 

< N/2. 

For all the j satisfying Inequality ^, if not half of them satisfy \i — j\ < N/2, we can use 
that new universal TM such that more than half of the j' satisfy \i — j'\ < N/2. And under 
the new universal TM, the j' satisfy Inequality |4] if j does. 

Now given i, index j can be specified in 

logvrd(i, j) 2 

bits. But since i is a corner point, we only need to enumerate 1/4 of the numbers, so to 
specify j, we really need only 



bits. Thus, 



hence 



log^vrd(i,i) 2 



log^7rd(i, j) 2 >C(j\i)> log N, 



d(i,j) > JanJk > ^8\i-j\/TT « y/2.5\i-j\ 
for iV/2c-many j's. 



□ 



Note, this applies to other distances (/oo and Manhattan) discussed in |lq ] as well. It is 
clear one can obtain a weaker average-case bound by consider any i instead of corner point 
i. 

Question: Can we improve this bound? Can we improve 2.5 to close to the worst-case 
3.5 constant factor in [16]? The upper bound is 4 given in Jl6|]. 
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5 Expected Length of a Longest Common Subsequence 



For two sequences (i.e. strings) s = s\ . . . s m and t = t\ . . .t n , we say that s is a subsequence 
of t if for some i\ < . . . < i m , Sj = t^.. A longest common subsequence (LCS) of sequences s 
and t is a longest possible sequence u that is a subsequence of both s and t. For simplicity, 
we will only consider binary sequences over the alphabet £ = {0, 1}. 

Let n be an arbitrary positive integer and consider two random strings s and t that 
are drawn independently from the uniformly distributed space of all binary string of length 
n. We are interested in the expected length of an LCS of s and t. Tight bounds on the 
expected LCS length for two random sequences is a well-known open question in statis- 



tics [17, pC| ]. After a series of papers, the best result to date is that the length is between 
0.762n and 0.838n [Q, ||, |6j, 0. The proofs are based on intricate probablistic and counting 
arguments. Here we give simple proofs of some nontrivial upper and lower bounds using 
the incompressibility method. 

Theorem 3 The expected length of an LCS of two random sequences of length n is at most 
0.867n + o(n). 

Proof. Let n be a sufficiently large integer. Observe that the expected length of 
an LCS of two random sequences of length n is trivially bounded between n/2 and n. 
By the Incompressibility Lemma, out of the 2 2n pairs of binary sequences of length n, at 
least (n — l)2 2n /n of them are log n-incompressible. Hence, it suffices to consider logn- 
incompressible sequences. 

Take a log n-incompressible string x of length 2n, and let s and t be the first and second 
halves of x respectively. Suppose that string u is an LCS of s and t. In order to relate the 
Kolmogorov complexity of s and t to the length of u, we re-encode the strings s and t using 
the string u as follows. (The idea was first introduced in fllOfl .) 

We first describe how to re-encode s. Let the LCS u = u\U2 ■ ■ ■ u m , where m = l(u). We 
align the bits of u with the corresponding bits of s greedily from left to right, and rewrite 
s as follows: 

s = a\U\a2U2 ■ ■ ■ a m u m s' . 

Here a\ is the longest prefix of s containing no ui, 0.2 is the longest substring of s following 
the bit u\ containing no U2, and so on, and s' is the remaining part of s after the bit u m . 
Thus o<i does not contain bit itj, for i = 1, . . . , m. In other words, each ctj is a unary string 
consisting of the bit complementary to U{. We re-encode s as string: 

s(u) = i(ai) 10 /(Q2) l---0 i(am) ls'. 

Clearly, given u we can uniquely decode the encoding s(n) to obtain s. 
Similarly, the string t can be rewritten as 

t = f3iUi(3 2 U2 ■ ■ ■ PmUmt', 

where each is a unary string consisting of the bit complementary to Uj, and we re-encode 
t as string: 

t(u) = 0^10^1 •••0^) It'. 

Hence, the string x can be described by the following information in the self-delimiting 
form: 
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1. A description of the above discussion. 

2. The LCS u. 

3. The new encodings s(u) and t(u) of s and t. 

Now we estimate the Kolmogorov complexity of the above description of x. Items 1 and 
2 take m + 0(l) bits. Since s(u) contains at least m l's, it is easy to see by simple counting 
and Stirling approximation (see e.g. (Tjl) that 



C(s(u)) < log £ M + 0(1) 



< < [l ) + o(i) 



< log n + log I n I + O(l) 



m 

< 2 log n + n log n — m log m — (n — m) log(n — m) + 0(1) 

The second step in the above derivation follows from the trivial fact that m > n/2. Similarly, 
we have 

C(t(u)) < 21ogn + n log n — to log m. — (n — m) log(n — m) + O(l) 

Hence, the above description requires a total size of 

O(logn) + m + 2nlogn — 2m log m — 2(n — m) log(n — m). 

Let p = n/m. Since C(x) >2n — logn, we have 

2n — log n < 0(log n) + m + 2n log n — 2m log m — 2{n — m) log(n — m) 
= 0(log n) + pn — 2np \ogp — 2n(l — p) log(l — p) 

Dividing both sides of the inequality by n, we obtain 

2 < o(l) +p — 2plogp — 2(1 — p) log(l — p) 

Solving the inequality numerically we get p < 0.867 — o(l). □ 

Next we prove that the expected length of an LCS of two random sequences of length 
n is at least 0.66666n — 0(y/n log n). To prove the lower bound, we will need the following 
greedy algorithm for computing common subsequences (not necessarily the longest ones). 

Algorithm Zero-Major(s = s± ■ ■ ■ s n , t = t±- ■ ■ t n ) 

1. Let u := e be the empty string. 

2. Let i := 1 and j := 1. 

3. Repeat steps 4-6 until i > n or j > n 



7 



4. If Si = tj then append bit Sj to string u; and i := i + j := j + 1 

5. Elseif Si = then j := j + 1. 

6. Else i := i + 

7. Return string u. 

Theorem 4 The expected length of an LCS of two random sequences of length n is at least 
0.66666n — 0(y/n logn). 

Proof. Again, let n be a sufficiently large integer, and take a log n-incompressible 
string x of length 2n. Let s and t be the first and second halves of x respectively. It suffices 
to show that the above algorithm Zero-Major produces a common subsequence u of length 
at least 0.66666n — 0{y/n logn) for strings s and t. 

The idea is to encode s and t (and thus x) using information from the computation of 
Zero-Major on strings s and t. We consider the comparisons made by Zero-Major in the 
order that they were made, and create a pair of strings y and z as follows. For each com- 
parison (sj, tj) of two complementary bits, we simply append a 1 to y. For each comparison 
(si,tj) of two identical bits, append a bit to the string y. Furthermore, if this comparison 
of identical bits is preceded by a comparison (si>,tjt) of two complementary bits, we then 
append a bit to the string z if i' = i — 1 and a bit 1 if f = j — 1. When one string (s or t) 
is exhausted by the comparisons, we append the remaining part (call this w) of the other 
string to z. 

As an example of the encoding, consider strings s = 1001101 and t = 0110100. Algo- 
rithm Zero-Major produces a common subsequence 0010. The following figure depicts the 
comparisons made by Zero-Major, where a "*" indicates a mismatch and a " I " indicates a 
match. 

s = 10 01101 

comparisons * | ** | | * | * 

t = 01101 

Following the above encoding scheme, we obtain y = 101100101 and z = 01100. 

It is easy to see that the strings y and z uniquely encode s and t and, moreover, 
l( y ) + l(z) = 2n. Since C{yz) > C(x)-21ogn > 2n-31ogn-0(l), and C{z) < I(z) + 0{1), 
we have 

C(y)>l(y)-3\ogn-0(l) 

Similarly, we can obtain 

C{z) > i(z)-31ogn-0(l) 

and 

C(w) > l(w) - 3 logn - 0(1) 

where w is the string appended to z at the end of the above encoding. 

Now let us estimate the length of the common subsequence u produced by Zero-Major 
on strings s and t. Let #zeroes(s) and #zeroes(t) be the number of 0's contained in s and 
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t respectively. Clearly, u contains m.in{#zeroes(s), #zeroes(t)} O's. From | ij (page 159), 
since both s and t are log n-incompressible, we know 

n/2 — 0{y/n log n) < #zeroes(s) < n/2 + 0(\/ n log n) 

n/2 — 0(yn logn) < #zeroes(t) < n/2 + 0(y/n logn) 

Hence, the string has at most 0(\/n logn) O's. Combining with the fact that C(w) > 
— 3 logn — 0(1) and the above mentioned result in pL we claim 

< 0(yn log n). 

Since Z(z) — i(io) = Z(it) , we have a lower bound on l(u): 

l(u) > l{z) - 0{^n logn). 

On the other hand, since every bit in the string y corresponds to a unique bit in the 
common subsequence u, we have l(u) > #zeroes(y). Since C{y) > l(y) — 2 logn — 0(1), 

K u ) — #zeroes(y) > l(y)/2 — 0{^/n logn). 

Hence, 

3l(u) > l(y) + l{z) - 0(v/nlogn) > 2n - O(Vnlogn). 

That is, 

l(u) > 2n/3 — O(ynlogn) ~ 0.66666n — 0(yjn logn) 

□ 

Our above upper and lower bounds are not as tight as the ones in [Q, [|, [| [jj. Recently, 
Baeza- Yates and Navarro improved our analysis and obtained a slightly better upper of 
0.860 [||. It will be interesting to know if stronger bounds can be obtained using the 
incompressibility method by more clever encoding schemes. 

6 Multidimensional Random Walks 

Consider a random walk in 1 dimension with fixed probability p = ^ of taking a unit 
step left or right. It is well-known that the maximal distance from the start position in 
either direction in a random walk of n steps is in the order of y/n with high probability. For 
example, the Law of the Iterated Logarithm, [O], says that the limit superior of this distance 

equals J ^n log log n with probability 1 for n rises unboundedly. Nonetheless, probabilistic 
analyses of random walks as in |9|, |l^] apparently are not concerned with flexible tradeoffs 
between probability and absolute upper or lower bounds on the largest distance traveled 
from the origin in every dimension as the theorem below. Such results however are very 
useful in theory of computation. 

In a random walk in k > 1 dimensions where each step increases or decreases the distance 
from the origin by a unit in exactly one dimension we would like to know the probability 
of traveling distance d from the origin in any dimension in n steps. 
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Theorem 5 Consider a random walk in k dimensions where each step is a unit step in 
any (but only one at a time) single dimension in positive or negative direction with uniform 
probability l/2k. Let <5(-) be a monotonic nondecreasing function and let x be a random 
walk of length n such that C(x\n) > n — 5(n). If n 3> k then the random walk x has all 
of the following properties (which therefore hold with probability at least 1 — l/2 5 ( n ) for a 
random walk of length n): 

(i) For every dimension, the maximal distance the walk moves away from the starting 
position in either direction during the walk is 0(y^(5(n) + log ^)); 

(ii) For every dimension, the maximum distance the walk is away from the starting 
position in either direction at the end of the walk is 0(^j5(n)j); and 

(iii) For every dimension, the minimum distance the walk is away from the starting 
position in either direction at the end of the walk is f2(y 2~ 5 ( n )^). 

(iv) For every dimension, the minimum distance the walk is away from the starting 
position in either direction at the end of an initial m-length segment x' with x = x'z for 

some z, C(x'\m) > m — 5(m), and m 3> k, is il(y / 2 _<5 ( m )^). 

Proof, (k = 1) For k = 1 we can identify left with and right with 1. So we are 
interested in the deviation of the relative frequency of l's in random walk x of length n. 

(i) If C(x\n) > n — 5(n) then for all n 

| #ones(x ) _ ™| < ^|(J( n )+0(l))n/loge (5) 

(Q)- 

The righthand side of Inequality |5| is an upper bound on the largest distance the random 
walk strays from the origin in either direction with probability > 1 — l/2 s ( n K This way we 
know the maximum distance from the origin at the end of a high-complexity walk of length 
n. 

(ii) We now analyze the maximum distance reached during the walk. A prefix y of length 
m < n of x has complexity C(y\m) > m — 5{n) — (1 + a) minjlog m, log(n — m)} + (9(l) where 
a is any fixed constant greater than 0. (Otherwise we can effectively describe x given n by a 
program p of length C{y\m) < m and a program r of length C(x\y) < n — m. To make one 
of them self-delimiting it suffices to add a prefix of length (1 + a) minjlog m, log(n — m)}.) 
Substituting in Inequality |5] and maximizing the minimum involved to n/2 we find 



\#ones{y) - — | < y -(o~(n) + (1 + a) logn + 0(l))m/log e (6) 

(iii) To obtain a lower bound on the largest distance the random walk necessarily strays 
from the origin in either direction during the walk we can do no better (using incompress- 
ibility) than determining the distance that necessarily exists in either direction at the end 
of a high-complexity random walk. Reference |ll| Lemma 2.6.2 on page 160 tells us that if 
C(x\n) > n — 5(n) then 

\#ones{x) - || > 2- & ^-°^^i. (7) 

The righthand side of Inequality is a lower bound on the largest distance the random walk 
strays from the origin in either direction with probability > 1 — 1 /2 &< ^ . 
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(iv) It is possible that a random walk x = x'z has an initial segment x' with l(x) = n 
and l(x') = m < n such that C(x\n) < n — i logn so that the number of O's can be equal 
to the number of l's while C(x'\m) > m — 5(m) (pj]]) so that the excess of O's over l's (or 
vice versa) is at least Q(V2~ s ( m ^m). 

(k > 1) For k > 1 we consider random walks as strings over the alphabet lo, li, • • • , 
ko,k\ where iq is a unit step backward in the i-dimension and i\ is a unit step forward. 
We first show (see Jll[| , p. 418) that if the overall string has high complexity, then also the 
subsequences over the {io, ^i}-alphabets have high complexity, for all i (1 < i < k). 

Claim 1 Let e > 0. Consider strings over an alphabet T, = {1,2, ... ,k}. For some i, 
denote the total number of occurrences of i's in y E X™ by m. If there is a constant 5 > 
such that 

C(y\k,n) > {n -5n 2e ) log k , (8) 

then |m — n/k\ < n l / 2+e . 

Proof. There are only D = (")(& — l) n ~ m strings x of length n with m occurrences 
out of i. Therefore, one can specify y by n,a,m and its index j, with l(J) = logD in this 
ensemble. An elementary estimate by Stirling's formula yields, for some 5 > 0, 



log (A; - l) n - m < (n - 5n Ie ) log A;. 



□ 



We can replace each element of i E £ by either i$ or to obtain a string x from y. We 
know that for every % E X the subsequence Xj of io> ii's of length in x satisfies 

C(^|y) < C(xi|mi) < mi + 0(l). 

Assume that for every i (1 < i < k) string Xi has randomness deficiency 5f. 

C(xi\y) >rrn- S^. 

From y,x\, . . . ,Xk we can reconstruct x and vice versa. Hence, since mj can be retrieved 
from y we can delimit the subprograms of length C(xi\y) by giving the length of m^ — C(xi\y) 
so that 

C(x\y) = C{xi,. . . ,x k \y) 
k 

< ^[C(x i |y) + 21og^] 

i=l 

k 



If 



< n-^[^-21og^]. 
C(x\y)>n-S(n), (9) 
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then 6(n) > YA=i[$i ~ 21og<5i] and therefore 

C{xi\y) > mi -5{n) - 2\og5{n) (10) 

for every i (1 < i < k). For every x G S" we have C(x\n,k) < nlog2& + 0(1), C(x|y) < 
n + 0(l) and C(y|fc,n) < n\ogk + 0{\). 
Choosing x such that 

C(x\k,n) > nlog2fc — 5(n) 

we have C(x| fc, n) > C(x|y) + C(y|A;, n) — 5{n) — O(l). We also have C(x\k, n) < C(x\y) + 
C(y\k, n) plus an additive term to encode the delimiter between the two constituents in the 
right-hand side. This additive term is logarithmic in the randomness deficiency of one of 
the terms. Therefore, both C(x\y) > n — 5(n) and C(y\k,n) > nlogk — 6(n) up to additive 
terms logarithmic in 5(n). Now both Inequalities || and || are satisfied simultaneously for 
5{n) = o(n) (Inequality |8| requires 6(n) < 5n 2e ). Then, by Claim || for every i (1 < i < k) 
the subsequence Xi of x over {io,«i} has length rrii = n/k ± n 1 / 2 and Inequality ^ holds. 
Therefore, the k > 1 case reduces to the case A; = 1 for every dimension simultaneously.^] 

Let #ij(x) denote the number of occurrences of ij in x. By our analysis of the 1- 
dimensional case this implies that the difference between the number of occurrences of io's 
and ii's during the random walk of length n is bounded above and below as follows: 



|#io(aO-#*i(aOI < y-(5(n)/2 + (l + a)logm i + 0(l))m i /loge 

= 0(^(W+log^)) 

This proves Items (i) and (ii). by Inequality ^ and assuming n S> k for the last equality. 
By Inequality it follows that 

|#io(x)-#»i(x)| > 2- 5 W/2-o(i)^- 

assuming n 3> k in the last equality. This proves Items (iii) and (iv). □ 

With this approach to random walks we can by varying the complexity of the walk 
(which implies varying the probability of such a walk in the sense that low complexity has 
high probability and higher complexity less probability) regulate the possible variation in 
the distance covered in the walk (high complexity walks have precisely fixed distance while 
low complexity walks have more uncertainty). 



6.1 Monopolist Game 



This approach is useful to solve the "Monopolist Game" defined in [21| as a formalization 
of a simplified version of a neural network updating rule due to von der Malsburg 111. This 
updating rule plays a key role in explaining orientation selectivity in the brain. 

This illustrates one of the advantages of the incompressibility argument: the single string that has 
high Kolmogorov complexity posesses every property simultaneously that can be proved. This contrasts 
with properties proven to hold with high probability: there only the strings in the intersection of the high 
probability subsets necessarily satisfies all such properties simultaneously. 
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In the Monopolist Game we start with k players that are given equal amounts 
I/k of the total initial capital of I units. The game is divided into discrete 
rounds. At every round one of the players wins and receives k — 1 units from 
the other players who each lose one unit. The players have equal probabilities 
1/k of winning a round. The game terminates if all but one player has lost all 
of the money. The surviving player has accumulated all the money and is called 
a monopolist. 

Denote the players by elements of E as in Claim [l|. Denoting just the winner in each round 
we can write the outcome of n rounds as x £ X™. 

Theorem 6 Consider the monopolist game with k, I , £, x,n as defined above and assume 
n 3> k. 

(i) For an outcome x with C(x\k, n) > nlog k — 5(k, n) with 5(k, n) = 5n 2e log k (5 > 
a constant), and hence with probability at least 1 — \/2 s ^ k ' n \ there is no monopolist for 

n</ 2/(l+2e)_ 

(ii) For an outcome x with C(x\n,k) > nlogfc — elogn (0 < e < -r) and hence with 
probability at least 1 — l/n e there is certainly a monopolist for some n satisfying n < 
(I/(k-2)) 2 /^l 

Proof. Let nj be the number of occurrences of i in x where l(x) = n. This way 
I/k + (k — l)ni — (n — rij) represents the capital of i at the end of x (positive or negative). 

(i) We are interested in the situation where there exist n' < n such that I/k + (k — 
Vjn'i - (n' - n<) < that is 

<-j<I (11) 

for k — 1 elements With C(x\k,n) > n — 5(k,n) and n < J 2 /( 1+2e ) Inequality |ll| is 

violated for every i £ S by Claim |l|. 

(ii) The number of strings in T, n with frequencies rij = n/k of symbol i (1 < i < k) is 
the multinomial coefficient satisfying 

n! 

= log 




nil • • -n k \ 

1 k 
~ nlog k — — {k — 1) log n + — log k + 0(1) 

using Stirling's approximation. This is the largest multinomial coefficient. Therefore, the 
logarithm of the total number of strings with frequencies |nj — n/k\ < / (n) is asymptotically 
upper bounded by 

1 k 

g(n,k) = nlogk - -(k - l)logn + - log k + (k— l)log/(n) + k + 0(1) (12) 

and we can describe every string of this ensemble by giving its index in at most that 
many bits. Consequently, if for some n-length outcome x of the monopolist game the 
complexity satisfies C(x\k,n) > g(n,k) then |nj — n/k\ < f{n). In particular, for n ^> k 
and C(x\n,k) > n log k — elogn we have f(n) < n^ 1 / 2 )^. 
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If — f( n ))(k — 1) — (n— t; — f(n)) > / then there is a monopolist by the nth round with 
certainty. This is the case if f(n) < I/(k — 2) which is the case if n = (I/(k — 2)) 2 ^ 1 ~ 2e \ 

□ 
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